Skip to content

add hook to automatically use ipython3 lexer on examples#59984

Closed
Aydin-ab wants to merge 4 commits intoray-project:masterfrom
Aydin-ab:docs/add-lexer-hook
Closed

add hook to automatically use ipython3 lexer on examples#59984
Aydin-ab wants to merge 4 commits intoray-project:masterfrom
Aydin-ab:docs/add-lexer-hook

Conversation

@Aydin-ab
Copy link
Contributor

@Aydin-ab Aydin-ab commented Jan 8, 2026

Issue

  • If publishing docs with a notebook, sphinx + myst-nb will usually default to a python3 lexer on the entire notebook unless the writer specifically set one in the notebook metadata (which frankly never happens)
  • The python3 lexer can not handle cells that starts with ! such as !pip install ... or other command cells.
  • This causes sphinx to throw a warning, making the CI fail because we've set Readthedocs to fail on warning
  • To silence the warning, publishers must manually set the notebook lexer to ipython3 that wraps around python3 and is able to parse command cells
  • This adds unnecessary overhead for publishers unfamiliar with Sphinx. They should only worry about their content, not what a lexer is

Ideally, sphinx should be able to pick a lexer for each cell but it only supports notebook-level lexer

Solution

This PR automatically adds a ipython3 lexer to any notebook listed in any of the glob patterns in ipython3_lexer_patterns. For more precision, they can also exclude files with other glob patterns in ipython3_lexer_exclude_patterns. This ensures:

  • Writers can build their notebook the way they want, using any type of cells (command, magic etc)
  • Flexibility in selecting which notebook are selected with an include/exclude pattern
  • Re-use sphinx compiler to keep it consistent with how sphinx uses glob pattern (for exclude_patterns for example)

Signed-off-by: Aydin Abiar <aydin@anyscale.com>
@Aydin-ab Aydin-ab marked this pull request as ready for review January 9, 2026 00:01
@Aydin-ab Aydin-ab requested a review from a team as a code owner January 9, 2026 00:01
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a Sphinx hook to automatically set the ipython3 lexer for specified notebook files. This is a great solution to prevent build failures caused by unsupported syntax in notebooks, like shell commands. The implementation is clean and correctly uses Sphinx's event system. I have a couple of suggestions to improve the code's robustness: one regarding more specific exception handling and another about adhering to file formatting conventions.

@Aydin-ab
Copy link
Contributor Author

Aydin-ab commented Jan 9, 2026

cc @aslonnie

@aslonnie aslonnie self-requested a review January 9, 2026 00:03
Signed-off-by: Aydin Abiar <aydin@anyscale.com>
@ray-gardener ray-gardener bot added docs An issue or change related to documentation core Issues that should be addressed in Ray Core labels Jan 9, 2026
Signed-off-by: Aydin Abiar <aydin@anyscale.com>
@Aydin-ab Aydin-ab requested a review from a team as a code owner January 9, 2026 22:25
Signed-off-by: Aydin Abiar <aydin@anyscale.com>
@github-actions
Copy link

This pull request has been automatically marked as stale because it has not had
any activity for 14 days. It will be closed in another 14 days if no further activity occurs.
Thank you for your contributions.

You can always ask for help on our discussion forum or Ray's public slack channel.

If you'd like to keep this open, just leave any comment, and the stale label will be removed.

@github-actions github-actions bot added the stale The issue is stale. It will be closed within 7 days unless there are further conversation label Jan 24, 2026
@github-actions
Copy link

github-actions bot commented Feb 7, 2026

This pull request has been automatically closed because there has been no more activity in the 14 days
since being marked stale.

Please feel free to reopen or open a new pull request if you'd still like this to be addressed.

Again, you can always ask for help on our discussion forum or Ray's public slack channel.

Thanks again for your contribution!

@github-actions github-actions bot closed this Feb 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Issues that should be addressed in Ray Core docs An issue or change related to documentation stale The issue is stale. It will be closed within 7 days unless there are further conversation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Comments